This invention relates to computer animation and visual effects. In particular, this invention relates to a method and apparatus for reproducing real-world lighting effects in a three dimensional computer animated object.
Visual effects used in movies and television programs have become extremely sophisticated in recent years. Older, more cumbersome methods of creating three dimensional physical models of fictitious creatures and objects and animating them on a scaled-down physical set through frame-by-frame motion manipulation have largely been replaced by the widespread availability of software for creating xe2x80x9cthree dimensionalxe2x80x9d computer models of animated objects, so-called because they exhibit surface qualities and lighting effects which simulate those of a three dimensional object. It is now commonplace to create a three dimensional computer model of an object and integrate the animated object as a visual effect into a cinematographic scene, which is a more realistic and more versatile method of creating visual effects creatures and objects.
Three dimensional computer-generated images of animated characters and objects can be modeled to be extremely realistic in appearance. The most popular computer programs presently available for creating three dimensional computer animated objects allow the animator to create a model of an object in three dimensions, specify the position of one or more synthetic light sources, and then apply basic rules of theoretical optics to generate lighting effects which mimic realistic three dimensional reflection patterns along the contours of the various surfaces of the model. This method has proven satisfactory for creating animated objects within an animated setting, because all of the images in the frame are subjected to the same synthetic light sources and all lighting effects are processed according to the same rules.
However, such a commercially available computer modeling programs cannot create a realistic visual effect object for integration into a real-world cinematographic scene. This is a much more difficult problem, because the computer animated object must blend seamlessly, and be consistent in all respects, with the images of the physical objects with which the visual effect will coexist in the cinematographic scene. Otherwise the visual effect will not be realistic and will lose its entertainment value to today""s sophisticated viewers.
One of the most important factors affecting the degree of realism obtained when an animated object is integrated with a real-world cinematographic object is consistency between the real-world and animated lighting effects. In order for the computer animated object to be perceived by a viewer as properly belonging within a real-world scene, synthetic lighting effects on the animated object must be perfectly consistent with the physical lighting effects that a comparable physical object would exhibit if placed on the physical set in the same position as the animated object. Even slight deviations in shadowing, reflection angles, coloring or light intensity are perceived by the viewer as anomalies, either consciously or unconsciously, and can significantly reduce or destroy the believability of the visual effect.
Conventional methods for integrating computer animated objects into real-world cinematographic objects involve first shooting the cinematographic scene on a physical set, with stationary and moving objects as they will appear in the finished cinematographic scene. The physical set and the objects in it are carefully illuminated by various types of light sources selected, positioned, oriented and often modified (eg. by dimmers and filters) according to the lighting effects desired to be achieved in each particular scene. A three dimensional model of the visual effects object is then created on a computer and processed by conventional modeling software, and the image is superimposed onto each frame of the cinematographic scene to create the composite cinematographic picture.
The basic optical parameters used in conventional three dimensional modeling software can be used to provide a starting point for the rendering of computer animated three dimensional objects for use as visual effects objects. However, the result is invariably unsuitable when the animated object is incorporated into a real-world scene because these parameters are developed from theoretical optics principles and do not factor in many real-world optical parameters. As such, the synthetic lighting of the animated object can never match the actual lighting effects applied to objects on the physical set as manifest in the cinematographic reproduction.
Thus, after incorporating the computer animated object into the cinematographic scene the animated object must be altered to be consistent with the lighting effects used on the physical set. This is a painstaking process which generally takes days or weeks of skilled visual effects artists adapting each shot, sometimes frame by frame, to match the lighting of each computer animated object with the lighting effects that would be exhibited by a comparable physical object on the physical set.
Apart from the tremendous cost involved in such an undertaking it is virtually impossible using this method, even for the most highly skilled visual effects artist, to achieve an exact reproduction of the lighting effects on the physical set so that the incorporation of an independently created computer animated object into the real-world cinematographic object is completely realistic. There are thousands of shades and hues of each colour in the spectrum, and while the human eye is capable of discerning only a few hundred at a conscious level smaller differences are noticeable as anomalies at an unconscious level.
This process is also monitor-dependent, to the extent that different computer monitors display slightly different colours, shades and hues, producing latent anomalies that can become apparent when the animated object is displayed on a different monitor or incorporated into a composite cinematographic scene. Differences between animated and real-world colours, shades and hues which may have been indistinguishable to visual effects artists, either due to the characteristics of the computer monitor used or due to the inability of the human eye to distinguish between fine differences in colours, shades and hues, can become apparent in the composite image and can destroy the believability of the visual effect.
This problem is exacerbated by the numerous factors which can affect the lighting of a cinematographic scene, including the positioning, orientation, luminous intensity and colour of each light source, changes in luminous intensity and colour due to filters, dimmers and light occlusion by opaque objects, diffusion and attenuation due to the position of the animated object, optical transmission characteristics of the camera lens, the sensitivity of the cinematographic film used in filming the scene, film colour balance, frame rates and shutter openings. All of these factors combine to create many diverse and unique lighting effects in a cinematographic scene, and while this diversity is useful to directors of lighting and cinematography in practising their craft, the resulting lighting effects can be extremely difficult even to approximate in a computer model, let alone duplicate.
It would accordingly be advantageous to provide a method and apparatus for applying lighting effects to a three dimensional computer animated object which are consistent in all respects with the lighting effects used on a real-world cinematographic set, such that upon integrating the animated object into the cinematographic scene to form a composite cinematographic image, the animated object is indistinguishable from a comparable physical object occupying the same position and performing the same motion as the computer animated object.
The present invention provides a method and apparatus for reproducing real-world (physical) lighting effects in a three dimensional computer model. The invention provides a means for quantifying photographic and cinematographic lighting parameters from physical light sources and cameras into photometric data, to create computer generated synthetic light sources that synthetically reproduce the physical lighting used on a physical set and are thus suitable for the rendering of three dimensional computer models that blend seamlessly into a cinematographic scene.
According to the invention, a three dimensional virtual set is created to a scale in proportion to the physical set. The position, orientation, and optical parameters of the physical light sources used on the real-world set are input into the model and adjusted to account for effects such as dimming, filtration, diffusion, occlusion and attenuation to produce synthetic light sources in the virtual set which mimic the physical light sources on the physical set. This procedure is repeated for each light source to create synthetic lighting corresponding to the lighting of the physical set. The lens transmission characteristics of the camera lens and sensitivity of the film may be incorporated into the synthetic lighting parameters, and the viewpoint is selected to correspond with the position of the camera. The animated object is positioned and oriented within the virtual set according to the position and orientation in which the visual effect is to appear in the cinematographic scene, and the synthetic light sources, adjusted to account for diffusion, attenuation and occlusion, are applied to the animated object.
The computer animated object is thus synthetically xe2x80x98illuminatedxe2x80x99 within the virtual set in exactly the manner that a physical object of the same shape in the physical set would have been illuminated, using measurements taken from the actual parameters used on the physical set. The invention thereby significantly reduces the labour and judgment required on the part of visual effects artists trying to reproduce the lighting effects in the composite scene ad hoc, by simulating the actual physical lighting and camera parameters as selected by the directors of lighting and photography. The invention accordingly substantially reduces the time required to integrate the visual effects object into a cinematographic scene, and the result is a visual effects object that is very realistic in appearance.
In the preferred embodiment optical parameters of various light sources, filters, cameras and films are maintained in a database or look-up table and can thus be applied to the virtual set by reference to the actual devices used on the real-world set. Other factors which are set-dependent, such as filtration, dimming and occlusion caused by opaque objects between the light source and the object position, are measured on the physical set and reproduced in the computer model. The visual effects artist need only position the animated object in each frame, and the computer applies the appropriate lighting effects to the surfaces of the animated object.
The present invention thus provides a method of reproducing in a three dimensional computer animated object lighting effects generated by light sources on a physical set, comprising the steps of inputting data corresponding to metric parameters of the physical set to generate a virtual set scaled in proportion to the physical set, inputting data corresponding to the positions and orientations of the light sources to generate synthetic light sources in the virtual set in positions and orientations corresponding to positions and orientations of corresponding physical light sources on the physical set, calculating effective luminous intensity and effective colour component parameters of the physical light sources, inputting data to position the computer animated object in the virtual set, and applying the effective luminous intensity and effective colour component parameters of the physical light source to the data.
The present invention further provides an apparatus for reproducing in a three dimensional computer animated object lighting effects produced by physical light sources on a physical set captured by a camera, comprising a computer having a computer monitor and software for generating data corresponding to metric parameters of the physical set to generate a virtual set scaled in proportion to the physical set, generating data corresponding to positions and orientations of the light sources to generate synthetic light sources in the virtual set in positions and orientations corresponding to positions and orientations of corresponding physical light sources on the physical set, calculating effective luminous intensity and effective colour component parameters of the physical light sources, generating a representation of the computer animated object in the virtual set, and applying the effective.
The present invention further provides a computer program product for use with a device for producing a representation of a three dimensional computer animated object and generating therein lighting effects corresponding to lighting effects produced by physical light sources on a physical set captured by a camera, the computer program product comprising computer readable program code for causing a computer to generate a representation of a virtual set scaled in proportion to the physical set, computer readable program code for causing a computer to generate synthetic light sources in the virtual set in positions and orientations corresponding to positions and orientations of corresponding physical light sources on the physical set, computer readable program code for causing a computer to calculate effective luminous intensity and effective colour component parameters of the synthetic light sources corresponding to effective luminous intensity and effective colour component parameters of the light sources, and computer readable program code for applying the effective luminous intensity and effective colour component parameters to the representation of the computer animated object to reproduce therein lighting effects corresponding to those of the physical set.